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Concept Mapping: Visualizing Student Understanding 

Abstract 

A pilot study to investigate the use of concept mapping for assessment purposes was conducted 
during the spring of 1996. A random sample of science teachers affiliated with the Iowa-Scope, Sequence, 
and Coordination Project (Iowa-SS&C Project) participated in the concept mapping study. Student concept 
maps were scored by the participating teachers using the Expert Science Teaching Educational Evaluation 
Model ( ESTEEM ) Concept Mapping Scoring Rubric. Even with attempts to standardize implementation of 
the concept mapping procedures, teachers had a tendency to divert from the pre-established guidelines. 
Concept mapping can be used for a diverse set of topics and issues and can be used across grade levels. 

Even with the use of a rubric, scoring concept maps is a subjective process. In addition to quantitative 
analyses of concept maps, qualitative inspection of concept maps can provide evidence of misconceptions 
and growth in student understanding. Since concept maps are a personal construction of understanding, a 
concept map would seem to warrant a personal, in-depth look. Numerous variables such as skill level with 
concept mapping, concept difficulty, different classrooms, and scoring of the maps are factors which make 
interpretations of the data problematic. 



Introduction 

According to Novak and Gowin (1984) a concept is a regularity in events or objects 
designated by some label. Concept maps are intended to represent meaningful relationships 
among concepts in the form of propositions. A proposition consists of two or more 
concept labels linked by words in a semantic network (Novak & Gowin, 1984). A concept 
map can be thought of as a schematic that represents a set of concept meanings embedded 
in a propositional framework. The concept map provides a schematic summary of learning 
that has occurred after a learning task has been completed. 

Concept maps should be hierarchical with more general, more inclusive concepts 
subsuming more specific concepts. Since most, if not all, concepts fit into a conceptual 
domain which consists of numerous interlinked pieces, concept maps, in addition to being 
a hierarchical representation, should also have cross-links that show interrelationships 
among the concepts on the map. Parts of the map may have linear representations or 
segments, but as conceptual understanding increases, cross-linking of the segment would 
be expected for a concept domain. Linking words added to connect and provide meaning 
to linkages on the map. Concept mapping is a technique for externalizing concepts and 
propositions (Novak & Gowin, 1984). 

In education, the challenge has been not only to help students elaborate conceptual 
understanding already possessed, but to modify those knowledge structures that contain 
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misconceptions or alternative conceptions of frameworks (Novak, 1990). Concept maps 
have also been recognized for their usefulness in the identification of student 
misconceptions. Using concept maps for assessment purposes presents challenges for 
both scoring and interpretation, and the activity of concept mapping also requires 
instruction and practice to become "fluent" in the act of setting concepts out on paper. 

When concept mapping, a student's conceptual understanding is translated into a graphic 
representation of that understanding onto a two dimensional space. This representation can 
be informally or formally assessed in an attempt to make an interpretation relative to the 
student’s conceptual understanding. 

Concept Mapping Pilot Study Description 

A pilot study to investigate the potential use of concept mapping for assessment 
purposes was conducted during the spring of 1996. A random sample of science teachers 
affiliated with the Iowa-Scope, Sequence, and Coordination Project (Iowa-SS&C Project) 
was invited to participate in the concept mapping study. This participatory invitation was 
also extended to all Iowa-SS&C teachers. 

Prior to the actual pilot study, a pre-pilot study was conducted by an Iowa-SS&C 
teacher who was experienced in using concept maps in the science classroom. The 
purposes of this pre-pilot study were to obtain input related to facilitating concept mapping 
instruction and to gain feedback from the use of the Expert Science Teaching Educational 
Evaluation Model ( ESTEEM ) Concept Mapping Scoring Rubric to score concept maps. 

The information from this preliminary study was incorporated in the pilot study. Appendix 
A: Using Concept Maps for Student Assessment was set out by the teacher who conducted 
the pre-pilot study to illustrate the ways in which he conceptualized the use of concept maps 
for student assessment. 

Before beginning the pilot study in their classrooms, in-service training in concept 
mapping and practice in the use of the ESTEEM Concept Mapping Scoring Rubric was 
provided at each of the project sites by Iowa-SS&C staff for the Iowa-SS&C teachers who 
would be participating in the study. Participant training included actual practice in 
constructing concept maps and training in the use of the ESTEEM Concept Mapping 
Scoring Rubric. The scoring procedures in the rubric were practiced, and this practice was 
accompanied by discussions that led to clarification and interpretation of score levels in the 
rubric. 

This application and discussion process of the training was intended to help achieve 
interrater agreement in interpretation and use of the ESTEEM rubric, since each teacher 
participant would be scoring their students' pretest and posttest concept maps according to 
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the rubric. Scoring concept maps is a time intensive process, and both pretest and posttest 
maps were to be scored by teachers using the ESTEEM Concept Mapping Scoring Rubric. 
The ESTEEM Concept Mapping Rubric Description 
The ESTEEM Concept Mapping Rubric consists of five categories: key and non- 
key words, lines, meaningful connections, meaningful segments, and meaningful total 
pattern. Using an oblique rotation, the five categories were assembled through a principal 
component factor analysis which resulted in five factors (categories) that accounted for 
86% of the variability (Burry-Stock, 1995). In the ESTEEM Manual, Burry-Stock notes 
that the rubric was developed using much of the work from Novak (1990) and Novak and 
Gowin (1984). A sample of the scoring scheme for pattern hierarchy which is a subpart of 
the category fifth category. Meaningful Total Pattern, is excerpted as an example of the 
rubric. An excerpt of Category 3: Meaningful Connections from the Esteem Concept 
Mapping Rubric follows. 

Category 3: Meaningful Connections 

Connecting lines are labeled with a word or symbol: 

5 Connecting lines are labeled with a word or symbol approximately 90% or more of the 

time. 

3 Connecting lines are labeled with a word or symbol approximately 70% or more of the 

time. 

1 Connecting lines are labeled with a word or symbol approximately 50% or less of the 

time. 

Category 3: Meaningful Connections 

Connections meaningful: 

5 The relationships between the concepts are meaningful approximately 90% or more of the 

time. 

3 The relationships between the concepts are meaningful approximately 70% or more of the 

time. 

1 The relationships between the concepts are meaningful approximately 50% or less of the 

time. 

Competency level descriptors are expert, proficient, competent, advanced beginner, and 
novice. Respective cut points for these levels are 85%, 70%, 35%, 15%, and 1%. The 
designation of these competency levels is based on the work of Berliner (1986) and 
Dreyfus and Dreyfus (1986) (cited in Burry-Stock, 1995). 

Pilot Study Participation 

Twenty-five Iowa-SS&C teachers, 13 of whom were from the random sample, 
participated in the study. Each participant in the pilot study selected one of their science 
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classes for participation in the study. Guidelines were provided to teachers for 
implementation of concept mapping with their selected class of students. Instruction and 
practice with concept mapping to facilitate students' skill with concept mapping was 
suggested. Teachers were asked to keep a log of their use of concept mapping with their 
classes. When the students had some skill level with concept mapping, students produced 
a concept map to be used as a pretest to the unit of study. 

The use of 12 to 15 key words that related to a concept or domain was suggested 
for both the pretest and posttest maps. Limiting the number of key words helped to keep 
the task of concept mapping more feasible for students who were inexperienced with 
concept mapping. The students could generate as many other words (non-key words) as 
they wished so were not limited to just using 12 to 15 words. Again, since scoring concept 
maps can be a very, time intensive process, for the purposes of the pilot study, the use of 
fewer key words also reduced some of the time associated with scoring the maps. 

Students were also asked to write a paragraph about the conceptual understandings 
represented on their posttest maps. This supporting, explanatory paragraph was requested 
to provide a comparison piece for the concept maps. Participating teachers were also asked 
to provide contextual information about the unit of study being completed by the classes. 

Concept Mapping Use for Assessment 

As evidenced from this pilot study, concept mapping can be used for a diverse set 
of topics and issues and can be used across grade levels. Participating classes ranged from 
grade 6 through grade 12 with multiple content areas being studied and concept mapped by 
students. A general categorization by content area for the maps would place the maps in 
life science/biology, physical science, or earth science. Biological topics included the 
respiratory, endocrine, and reproductive systems, blood, heredity and DNA, life origins 
and evolution, Jurassic Park, AIDS, invertebrates, and plants. Electricity and light were 
physical science areas, and earth science topics such as weather, rocks, water, and Big 
Bang were used by other classes. This diversity is in keeping with the constructivist/STS 
(Science/Technology/Society) underpinnings of SS&C. Students and teachers were 
encouraged to make interdisciplinary connections as well as studying science that has local, 
relevant, and personal implications. 

The guidelines which had been established in an attempt to standardize the pilot 
study were implemented by the teachers in a variety of ways. While most of the teachers 
returned requested materials, some of the teachers did not provide contextual information or 
did not return student summary paragraphs. The extensiveness and quality of the logs also 
varied. In a preliminary analysis of the score sheets for the pretests and posttests, 
problems with scoring procedures using the rubric have been noted. Concept maps are 
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being scored by another set of raters to look not only for data errors, but for interrater 
agreement in use of the ESTEEM rubric and revisions for clarity in the rubric. 

Classroom teachers did vary the approach to concept mapping and application of the 
rubric, perhaps scoring some rubric categories of maps and not others. While the 
classroom teacher can score and interpret concept mapping to meet their instructional and 
assessment purposes, this practice limits the ability to make inferences about students' 
conceptual understanding for study purposes. Without control of numerous variables 
across classrooms, ambiguity in interpreting levels of performance in a classroom is 
difficult. Given just the variability in implementation of concept mapping alone, the issues 
of validity and reliability are also of major importance when using concept maps for 
assessment purposes (Ruiz-Primo & Shavelson, 1996; Shavelson, Lang, & Lewin, 1994). 

Anderson & Huang (1989) (as cited in Shavelson, Lang, & Lewin, 1994)) reported 
correlations between concept map scores and measures of achievement and ability. A 
corrrelation of 0.69 was reported on an essay test on a unit and a corresponding concept 
map. The correlation with school science grades was 0.49. The correlation between 
concept maps and the Stanford Science Achievement Test was 0.66, and the correlation 
between concept maps and the Otis Lennon School Ability Test was 0.74. Since concept 
mapping does require practice to become proficient with just the skill of mapping, this 
should be kept in mind when making inferences or interpreting correlational data. The 
difficulty of the concepts and the mastery of concepts would also need to be considered 
when making interpretations about map scores. Concept maps can be one of multiple 
instructional or assessment tools that may elicit a kind of student performance not captured 
by other assessments. 

Qualitative and Quantitative Student and Teacher Data 

In Biology Students' Words 

Related to a unit of study on invertebrates, students in a biology class had these 
comments about using concept mapping. Not all of the 26 students commented on concept 
mapping, and the comments that follow are a sample of comments. No particularly 
negative comments were made by students. Students were asked to write summary 
paragraphs about the posttest concept map and some of these excerpted comments follow. 

Although some of the lines may have crossed and made everything a little harder to read, I 
felt I learned more doing it this way. It gave us a chance to see what we learned instead of 
giving the book answers to the book's questions. It made people think a little harder and 
prepare a bit better. 

This was a good way of learning because it showed many things about each class and 
connected them together. When doing a concept map, you can get down to the very 
smallest details. It 's neat to learn about different behaviors among the invertebrates. I 
learned many different things about stuff I knew nothing about. 
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We started with twenty key words that we had to use. We also had to add at last seven 
more key words of our own. That was very easy once you got going because you could 
expand in so many ways on the map. By my ending map you can tell I now know a lot 
more. 

My map represented to me what I learned in the class. I knew more about arthropods and 
my map reflected it. I didn’t know much about segmented worms and my map reflected 
that. I like them (maps) because if you know a lot about one subject and not alot about 
another you can still get an A. My map was what I remembered about invertebrates. 

I liked doing this concept map because I think the teacher gets a better idea of what the 
student knows, rather than knowing to fill in ’a’, ’b\ ’c’, or ’d\ It also is more fair to the 
student because they can really actually study for the correct thing, rather than studying 
for a multiple choice test, that of which some questions are the dumbest, knit picky 
questions that are almost completely irrelevant to the topic. I liked doing the concept 
map. 

It was supposed to be fun and test our creative side of our brain. 

... I can’t believe how much I know now. I think concept maps makes it easier to learn 
and understand biology. 

My concept map represents what I have learned and how I can relate ideas to each other. 

This is my favorite way of learning, because I can learn what I want and then I can 
express what I learned to someone else. This is the fairest kind of test for me because I 
can tell what I know and don't have to worry about what I don’t. 

I think this was a good idea it helped me better understand invertebrates. 

When I first began to learn about a concept map, I thought it would be difficult and very 
boring. Little did I know, everyday I use some type of concept map. Using a concept 
map is like using an outline when you’re writing. Using a concept map is also a way to 
keep everything organized. 

The concept map is a good quiz because you can put what you know and it's done 
independently. 

The concept map made it easier for me to explain things, instead of trying to guess what 
is going to be on the test. 

A Biology Teacher's Comments 

The teacher of this biology class noted that the topics in the unit of study on 
invertebrates were very difficult for the students in this class, and that students had 
previously worked with the concepts in this unit of study in a superficial way. This was 
the first experience with concept mapping in this class, and the teacher noted that on a scale 
of from "1" being not skilled in the use of concept mapping to a "5" being highly skilled, 
that a "2" represented this teacher’s perceived skill with concept mapping. This teacher 
also used the ESTEEM rubric with the students as they were learning to concept map. The 
teacher worked with the students to help them provide examples of what a 5, 3, or 1 might 
"look like" when scored with the rubric. 
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A Middle School Concept Mapping Example of the Respiratory System 

Students in a seventh grade middle school science class concept mapped their unit 
of study on the respiratory system. The teacher's goals for the unit of study were to have 
students 1) define respiration; 2) understand the structure and function of the respiratory 
system; 3) be able to explain lung function in excretion; and 4) understand the effects of 
diseases on the respiratory system. Figure 1 and Figure 2 are stem-and -leaf plots of the 
pretest and posttest data from this teacher's classroom. The pretest score range was from 
16 to 35 with a median of 24, and posttest scores ranged from 18 to 36 with a median of 
23. 

The t-value of 1 .466 from a contrast of pretest and posttest map scores was not 
significant (p=0.156) as shown in Table 1. Given the nature of concept maps, rather than 
leaving the data analysis when no significance was noted for this set of maps, qualitative 
evidence of student understanding does exist in the maps. A pretest and posttest 
comparison of the maps can serve to provide evidence of misconceptions, rearrangement 
and the addition of new information to the map can serve as evidence of student 
understanding. If the pretest and posttest maps have not changed substantially, this too 
would raise questions for instruction. 

Regis, Albertazzi, & Roletto (1996), in their discussion of the use of concept maps 
in chemistry over the course of four years, note that concept maps are highly idiosyncratic 
representations of domain-specific knowledge, and the interindividual differences among 
the maps were far more striking than the similarities. Their shift in emphasis in scoring 
moved to a focus on changes in content and organization of concept maps over time. The 
maps also were used to raise student awareness of their own and others' understandings. 
Concept maps could serve to be effective tools in both interpersonal and extrapersonal 
aspects of learning. 

Summary 

From the pilot study, concept mapping appears to be an instructional and 
assessment tool that can be used by students. Students' comments about the use of concept 
mapping tend to be favorable, and concept mapping can empower the student in the 
assessment process. Students, from their comments, appear to feel more ownership of 
what they must do to demonstrate their learning and understanding. 

Mechanically, concept mapping presents a somewhat difficult task in terms of 
interpretation and scoring of maps. The scores based upon a rubric do provide information 
about representation of student understanding, but the information represented on the 
concept map actually seems to necessitate more than a score based upon a rubric. Since the 
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construction of understanding is unique to an individual, it would seem that analyzing maps 
necessitates accompanying analyses for interpreting the map. 

Information gained from this pilot study that will be used in the design of other 
such studies suggests that other sources of information from the teacher and student would 
be useful in interpretation of the maps. A more in-depth set of lesson plans for a unit may 
help to provide a basis to look for changes in pretest and posttest maps. Classroom 
observations and student interviews about their represented understandings could provide 
insight for interpretation of intended meanings. Corroborating evidence from other test 
formats could also be an aid in making comparisons of concept maps to other formats of 
assessing understanding. 



References 

Burry-Stock, J. A. (1995). Expert science teacher educational evaluation model 
( ESTEEM )‘. Theory, development and research. Tuscaloosa, AL: The University of 
Alabama. 

Novak, J.D. (1990). Concept mapping: A useful tool for science education. Journal 
of Research in Science Teaching, 27,(10), 937-949. 

Novak, J.D., & Gowin, D.R. (1984). Learning how to learn. New York: 
Cambridge Press. 

Regis, A., Albertazzi, P.G., & Roletto, E. (1996). Concept maps in chemistry 
education. Journal of Chemical Education, 73,(11) 1054-1088. 

Ruiz-Primo, M.A., & Shavelson, R.J. (1996). Problems and issues in the use of 
concept maps. Journal of Research in Science Teaching, 33,(10), 569-600. 

Shavelson, R.J., Lang, H., & Lewin, B. (1994). On concept maps as potential 
"authentic" assessments in science. (CSE Tech. Rep. No. 388). Los Angeles, CA: 
University of California, National Center for Research on Evaluation, Standards, and 
Student Testing (CRESST). 




10 



Concept Mapping 9 



« 



Figure 1. 

Stem-and-Leaf Plot of Pretest 
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Figure 2. 

Stem-and-Leaf Plot of Posttest 
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Table 1 

Contrast for Pretest and Posttest Concept Maps (N = 25) 



Mean SD 

Pretest Map 23.44 4.74 

Posttest Map 24.64 5.26 

Difference 1.20 4.09 



The t-value of 1.466 was not significant. p=0.156 
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